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Remarks 

This Reply is in response to the Office Action mailed November 12, 2008. 

I. Summary of Examiner's Rejections 

In the Office Action dated November 12, 2008, Claim 10 was objected to for various 
informalities. Claims 36-45 were rejected under 35 U.S.C. 101 as being directed to non-statutory 
subject matter. Claims 1-5, 9-14, 16-17 and 36-46 were rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ludwig et al. (U.S. Patent No. 7,185,054, hereafter Ludwig) in view of Okuno 
(U.S. Patent No. 6,977,672) and further in view of Saka (U.S. Patent Publication No. 
2004/0070608). Claims 6, 8, and 15 were rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ludwig, in view of Okuno and Saka, and further in view of Shneiderman (U.S. Patent No. 
7,010,751). Claim 7 was rejected under 35 U.S.C. 103(a) as being unpatentable over Ludwig in 
view of Okuno and Saka, and further in view of Emens et al. (U.S. Patent No. 6,463,343, hereafter 
Emens). Claims 18 and 21 were rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ludwig in view of Shneiderman. Claims 19-20 and 22-23 were rejected under 35 U.S.C. 103(a) as 
being unpatentable over Ludwig in view of Shneiderman, and further in view of Okuno. Claims 24, 
26-27, 30, 32, and 34-35 were rejected under 35 U.S.C. 103(a) as being unpatentable over Emens 
in view of Ayatsuka et al. (U.S. Patent No. 7,188,139, hereafter Ayatsuka), and further in view of 
Burt et al. ("Object tracking with a moving camera," IEEE An Application of Dynamic Motion 
Analysis, 1989, pp 2-12, hereafter Burt). Claim 25 was rejected under 35 U.S.C. 103(a) as being 
unpatentable over Emens in view of Ayatsuka and Burt, and further in view of Andersson (U.S. 
Patent Publication No. 2002/0111999 A1). Claim 28 was rejected under 35 U.S.C. 103(a) as being 
unpatentable over Emens in view of Ayatsuka and Burt, and further in view of Hildebrandt (U.S. 
Patent Publication No. 2004/0070616). Claim 33 was rejected under 35 U.S.C. 103(a) as being 
unpatentable over Emens in view of Ayatsuka and Burt, and further in view of Vetterli et al. (U.S. 
Patent Publication No. 2002/0075282, hereafter Vetterli). Claim 29 was rejected under 35 U.S.C. 
103(a) as being unpatentable over Emens in view of Ayatsuka, Burt, and Andersson, and further in 
view of Westfield (U.S. Patent No. 6,677,979). Claim 31 was rejected under 35 U.S.C. 103(a) as 
being unpatentable over Emens in view of Ayatsuka and Burt, and further in view of Westfield. 

II. Summary of Applicant's Amendments 

The present Reply amends Claims 1, 10, 18, 24, and 37-46; and cancels Claims 33 and 36, 
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leaving for the Examiner's present consideration Claims 1-32, 34-35, and 37-46. Reconsideration 
of the Application, as amended, is respectfully requested. 

III. Claim Objections 

In the Office Action dated November 12, 2008, Claim 10 was objected to for various 
informalities. Accordingly, Claim 10 has been amended as shown above. Applicant respectfully 
submits that Claim 10 now conforms to Examiner's objections and reconsideration thereof is 
respectfully requested. 

IV. Claim Rejections Under 35 U.S.C. 5 101 

In the Office Action dated November 12, 2008, Claims 36-45 were rejected under 35 U.S.C. 
101 as being directed to non-statutory subject matter. Accordingly, Claims 37-45 have been 
amended, as shown above. Applicant respectfully submits that Claims 37-45, as amended, 
conform to the requirements of 35 U.S.C. 101 and reconsideration thereof is respectfully requested. 

Claim 36 has been canceled, rendering moot the rejection of this claim. 

V. Claim Rejections Under 35 U.S.C. § 103 
Claims 1-17 and 36-46 

In the Office Action dated November 12, 2008, Claims 1-5, 9-14, 16-17 and 36-46 were 
rejected under 35 U.S.C. 103(a) as being unpatentable over Ludwig (U.S. Patent No. 7,185,054) in 
view of Okuno (U.S. Patent No. 6,977,672) and further in view of Saka (U.S. Patent Publication No. 
2004/0070608). Claims 6, 8, and 15 were rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ludwig, in view of Okuno and Saka, and further in view of Shneiderman (U.S. Patent No. 
7,010,751). Claim 7 was rejected under 35 U.S.C. 103(a) as being unpatentable over Ludwig in 
view of Okuno and Saka, and further in view of Emens (U.S. Patent No. 6,463,343). 

Claim 1 

Claim 1 has been amended to more clearly define the embodiment therein. As amended, 
Claim 1 defines: 

1. (Currently Amended) A method for exchanging information in a shared interactive 
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environment, comprising: 

selecting a first physical device in a first live video image wherein the first physical 
device has information associated with it; 

causing the information to be transferred to a second physical device in a second live 
video image wherein the transfer is brought about by manipulating a visual representation of 
the information in the first live video image and in the second live video image; 

wherein at least one of the first physical device and the second physical device has a 
statically or dynamically defined hotspot in the first live video image or the second live video 
image; 

wherein the manipulation includes interacting with the first live video image and the 
second live video image; 

wherein the first physical device and the second physical device are part of the 
shared interactive environment; and 

wherein the first physical device and the second physical device are not the same. 

Claim 1 defines a method for exchanging information in a shared interactive environment. 
The method comprises selecting a first physical device in a first live video image wherein the first 
physical device has information associated with it; and causing the information to be transferred to 
a second physical device in a second live video image wherein the transfer is brought about by 
manipulating a visual representation of the information in the first live video image and in the 
second live video image. At least one of the first physical device and the second physical device 
has a statically or dynamically defined hotspot in the first live video image or the second live video 
image. The manipulation includes interacting with the first live video image and the second live 
video image. The first physical device and the second physical device are part of the shared 
interactive environment. The first physical device and the second physical device are not the same. 

Ludwig discloses a teleconference system for conducting a teleconference among a plurality 
of participants. The system has a plurality of video display devices, each having associated 
participant video capture capabilities and participant audio capture and reproduction capabilities. 
(Abstract). 

Okuno discloses an information control system capable of executing a desired action in 
accordance with the angle position of a camera. (Abstract). Assume that panning, tilting, and 
zooming of the camera 115 are controlled, and a printer 21 1 connected to a network 201 is image- 
sensed as a main object by the camera 115, as shown in FIG. 18. In this case, as a program is 
activated, a window 38 pops up in the display screen 36 of the display unit 104, and the user can 
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operate and set the printer 21 1 by using the window 37. (Column 4, lines 35-41 ). 

Saka discloses an apparatus and method for transferring files between a plurality of 
computers in a virtual network. (Abstract). Moreover, the present invention includes a more user- 
friendly display that allows a user to transfer a file between the two computers by simply moving the 
file icon from the second computer desktop to the adjacent computer desktop. (Paragraph [001 1]). 

Applicant respectfully submits that the cited references, alone or in combination, do not 
disclose or render obvious the embodiment of Claim 1 , as amended. Claim 1 has been amended to 
more clearly define causing the information to be transferred to a second physical device in a 
second live video image wherein the transfer is brought about by manipulating a visual 
representation of the information in the first live video image and in the second live video image. 

As described above, Ludwig appears to disclose a teleconference system showing live 
images of each participant. Okuno appears to disclose a system, including a camera, that can 
identify objects in view of the camera and enable a user to control the objects. Saka appears to 
disclose a system for transferring files between networked computers using graphical 
representations of each computer's file system. 

Applicant respectfully submits that none of the references appear to disclose manipulating 
any information in live video images. Ludwig merely shows multiple live views for a teleconference 
and Okuno appears to launch a separate program in a separate window to control objects in view of 
the camera. Saka discloses transferring files between file systems. Claim 1, however, defines 
causing the information to be transferred to a second physical device in a second live video image 
wherein the transfer is brought about by manipulating a visual representation of the information in 
the first live video image and in the second live video image (emphasis added). Furthermore, Saka 
appears to teach away from transferring information across live video images. Instead of using live 
video images, Saka uses graphical representations of each computer's file system to transfer 
information. Applicant respectfully submits that Ludwig, in view of Okuno and Saka, does not 
disclose or render obvious the embodiment of Claim 1 . 

Additionally, Claim 1 has been amended to define that at least one of the first physical 
device and the second physical device has a statically or dynamically defined hotspot in the first live 
video image or the second live video image. Applicant respectfully submits that the cited 
references, alone or in combination, do not disclose or render obvious this feature. 

In view of the above comments, Applicant respectfully submits that Claim 1, as currently 
amended, is neither anticipated by nor obvious in view of the cited references, and reconsideration 
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thereof is respectfully requested. 
Claims 10, 37, and 46 

The comments provided above with respect to Claim 1 are hereby incorporated by 
reference. Claims 10, 37, and 46 have been similarly amended to more clearly define the 
embodiments therein. For similar reasons as provided above with respect to Claim 1, Applicant 
respectfully submits that Claims 10, 37, and 46, as amended, are likewise neither anticipated by, 
nor obvious in view of the cited references, and reconsideration thereof is respectfully requested. 

Claims 2-9, 11-17, and 38-45 

Claims 2-9, 1 1-17, and 38-45 depend from and include all of the features of Claims 1, 10, or 
37. Claims 2-9, 11-17, and 38-45 have not been addressed separately but it is respectfully 
submitted that these claims are allowable as depending from an allowable independent claim, and 
further in view of the comments provided above. Reconsideration thereof is respectfully requested. 

Claim 36 

Claim 36 has been canceled, rendering moot the rejection of this claim. 
Claims 18-23 

In the Office Action dated November 12, 2008, Claims 18 and 21 were rejected under 35 
U.S.C. 103(a) as being unpatentable over Ludwig in view of Shneiderman. Claims 19-20 and 22-23 
were rejected under 35 U.S.C. 103(a) as being unpatentable over Ludwig in view of Shneiderman, 
and further in view of Okuno. 

Claim 18 

Claim 18 has been amended to more clearly define the embodiment therein. As amended, 
Claim 18 defines: 

18. (Currently Amended) A method for annotating a live video image, comprising: 
annotating the live video image; 

automatically transferring the annotation to a physical device in the live video image 
if the annotation is at least partially drawn over the physical device as it appears in the live 
video image; and 
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wherein the annotation is visible to at least one participant in a shared interactive 
environment. 

Claim 18, as amended, defines a method for annotating a live video image. The method 
comprises annotating the live video image. The method also comprises automatically transferring 
the annotation to a physical device in the live video image if the annotation is at least partially drawn 
over the physical device as it appears in the live video image. The annotation is visible to at least 
one participant in a shared interactive environment. 

Ludwig discloses a teleconference system for conducting a teleconference among a plurality 
of participants. (Abstract). Ludwig also discloses that FIG. 2B illustrates data shared and 
annotated by those conferees (lower left window). (Column 6, lines 59-61; Figure 2B). 

Shneiderman discloses that [Shneiderman's] invention addresses the problem of annotating 
commercial and/or personal electronic images, by providing software that permits users to easily 
accomplish such annotation, through the drag-and-drop of annotations from a predefined, but 
extendable, list. The annotations are placed at an X,Y location on the image determined by the 
user, and stored in a searchable database. (Column 4, lines 23-29). Shneiderman also discloses 
that the term "electronic image," as used herein is intended to include photographs, video, 
drawings, writings, webpages, paintings, holographic images, cartoons, sketches, renderings, etc. 
that can be stored electronically. (Column 7, lines 29-32). 

Claim 18 has been amended to more clearly define automatically transferring the annotation 
to a physical device in the live video image if the annotation is at least partially drawn over the 
physical device as it appears in the live video image. Applicant respectfully submits that Ludwig in 
view of Shneiderman does not disclose or render obvious this feature. 

As described above, Ludwig, in view of Shneiderman, appears to disclose annotating data 
that is stored electronically, such as an image. In Ludwig, participants in a teleconference appear 
to share data that they can annotate during the teleconference. In Shneiderman, a user can 
annotate electronic data, such as an image, from a selection of predefined annotations. Both 
references appear to disclose annotating some form of electronically stored data, however, neither 
reference, alone or in combination, appears to disclose or render obvious automatically transferring 
the annotation to a physical device in the live video image. 

In both references, the annotation appears to be limited to the electronically stored data. 
For example, in Shneiderman, the annotations are made to a picture of people at a conference. 
This is shown in more detail in Figure 7, which shows the picture annotated to identify each person. 
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Neither reference appears to show a physical device in a live image, instead both references only 
appear to show people. Additionally, the annotations do not appear to be transferred to any 
physical device in the live image. Instead, the annotations appear to remain fixed to whatever data 
was annotated. Applicant respectfully submits that Ludwig, in view of Shneiderman, does not 
disclose or render obvious the embodiment of Claim 1 8. 

In view of the above comments, Applicant respectfully submits that Claim 18, as currently 
amended, is neither anticipated by nor obvious in view of the cited references, and reconsideration 
thereof is respectfully requested. 

Claims 19-23 

Claims 19-23 depend from and include all of the features of Claim 18. Claims 19-23 have 
not been addressed separately but it is respectfully submitted that these claims are allowable as 
depending from an allowable independent claim, and further in view of the comments provided 
above. Reconsideration thereof is respectfully requested. 

Claims 24-35 

In the Office Action dated November 12, 2008, Claims 24, 26-27, 30, 32, and 34-35 were 
rejected under 35 U.S.C. 103(a) as being unpatentable over Emens in view of Ayatsuka (U.S. 
Patent No. 7,188,139), and further in view of Burt ("Object tracking with a moving camera," IEEE An 
Application of Dynamic Motion Analysis, 1989, pp 2-12). Claim 25 was rejected under 35 U.S.C. 
103(a) as being unpatentable over Emens in view of Ayatsuka and Burt, and further in view of 
Andersson (U.S. Patent Publication No. 2002/0111999 A1). Claim 28 was rejected under 35 U.S.C. 
103(a) as being unpatentable over Emens in view of Ayatsuka and Burt, and further in view of 
Hildebrandt (U.S. Patent Publication No. 2004/0070616). Claim 29 was rejected under 35 U.S.C. 
103(a) as being unpatentable over Emens in view of Ayatsuka, Burt, and Andersson, and further in 
view of Westfield (U.S. Patent No. 6,677,979). Claim 31 was rejected under 35 U.S.C. 103(a) as 
being unpatentable over Emens in view of Ayatsuka and Burt, and further in view of Westfield. 
Claim 33 was rejected under 35 U.S.C. 103(a) as being unpatentable over Emens in view of 
Ayatsuka and Burt, and further in view of Vetterli (U.S. Patent Publication No. 2002/0075282). 

Claim 24 

Claim 24 has been amended to more clearly define the embodiment therein. As amended, 
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Claim 24 defines: 

24. (Currently Amended) A shared interactive environment, comprising: 

a camera system to provide a first live view and a second live view different from the 
first live view, wherein the second live view can be configured to zoom in on a portion of the 
first live view; 

a first graphical user interface (GUI) coupled to the camera system and to present 
the first live view and the second live view, wherein the views can capture a physical device; 

a device controller to dynamically control the physical device in response to 
interaction of a first user with the GUI wherein the interaction can including annotating at 
least one of: 1) the first live view; and 2) the second live view; 

wherein annotations are automatically transferred to the physical device in the live 
views if the annotation is at least partially drawn over the physical device as it appears in the 
live video image; 

a device tracker coupled to the camera system and to dynamically recognize new 
physical devices; and 

wherein the camera system can be mounted on a mobile, robotic platform. 

Claim 24, as amended, defines a shared interactive environment. The shared interactive 
environment comprises a camera system to provide a first live view and a second live view different 
from the first live view, wherein the second live view can be configured to zoom in on a portion of 
the first live view. The shared interactive environment also comprises a first graphical user 
interface (GUI) coupled to the camera system and to present the first live view and the second live 
view, wherein the views can capture a physical device. Additionally, the shared interactive 
environment comprises a device controller to dynamically control the physical device in response to 
interaction of a first user with the GUI wherein the interaction can including annotating at least one 
of: 1) the first live view; and 2) the second live view. Annotations are automatically transferred to 
the physical device in the live views if the annotation is at least partially drawn over the physical 
device as it appears in the live video image. The shared interactive environment further comprises 
a device tracker coupled to the camera system and to dynamically recognize new physical devices. 
Additionally, the camera system can be mounted on a mobile, robotic platform. 

Emens discloses a method for controlling a remote device from a client computer using a 
digital image of a remote location associated with the remote device. (Abstract). 

Ayatsuka discloses that each device arranged in an information space and a computer are 
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sustainedly connected and the connection can be visually recognized on the display screen of the 
computer. (Abstract). 

Burt discloses that the task of detecting and tracking moving objects is particularly 
challenging if it must be performed with a camera that is itself moving. Yet, in applications such as 
automated surveillance and navigation, this task must be performed continuously, in real time, and 
using only modest computing hardware. (Abstract). 

Claim 24 has been amended to define wherein the interaction can including annotating at 
least one of: 1) the first live view; and 2) the second live view. Claim 24 has also been amended to 
define wherein annotations are automatically transferred to the physical device in the live views if 
the annotation is at least partially drawn over the physical device as it appears in the live video 
image. Applicant respectfully submits that Emens, in view of Ayatsuka and Burt, does not disclose 
or render obvious these features. 

Additionally, the amendments to Claim 24 include features similar to those of Claim 33 as 
originally presented. Claim 33 was rejected as unpatentable over Emens, in view of Ayatsuka and 
Burt, further in view of Vetterli. Vetterli appears to disclose annotating "a view, e.g. of scenery, of a 
shopping or museum display, or of a meeting or conference." (Abstract). Vetterli does not appear 
to disclose transferring the annotations to the physical device in the live views. Instead, the 
annotations appear to be made only to the image and not transferred to any device shown in the 
image. Applicant respectfully submits that Emens, in view of Ayatsuka and Burt, and further in view 
of Vetterli does not disclose or render obvious the embodiment of Claim 24, as amended. 

In view of the above comments, Applicant respectfully submits that Claim 24, as currently 
amended, is neither anticipated by nor obvious in view of the cited references, and reconsideration 
thereof is respectfully requested. 

Claims 25-32 and 34-35 

Claims 25-32 and 34-35 depend from and include all of the features of Claim 24. Claims 25- 
32 and 34-35 have not been addressed separately but it is respectfully submitted that these claims 
are allowable as depending from an allowable independent claim, and further in view of the 
comments provided above. Reconsideration thereof is respectfully requested. 

Claim 33 

Claim 33 has been canceled, rendering moot the rejection of this claim. 
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VI. Conclusion 

In view of the above amendments and remarks set forth above, it is respectfully submitted 
that all of the claims now pending in the subject patent application should be allowable, and 
reconsideration thereof is respectfully requested. The Examiner is respectfully requested to 
telephone the undersigned if he can assist in any way in expediting issuance of a patent. 

The Commissioner is authorized to charge any underpayment or credit any overpayment to 
Deposit Account No. 06-1325 for any matter in connection with this response, including any fee for 
extension of time, which may be required. 

Respectfully submitted, 



Date: March 12. 2009 By: /Nathan L Feld/ 

Nathan L. Feld 
Reg. No. 59,725 



Customer No. 23910 
FLIESLER MEYER LLP 
650 California Street, 14 th Floor 
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